Summary

Examining the impact of cognitive load on structure learning

6 agents, 12 issues

Method changes:

  • ?
Demographics (Attention Check)
0
0.25
0.5
0.75
1
Overall
high
(N=21)
high
(N=22)
high
(N=22)
high
(N=15)
high
(N=15)
high
(N=95)
age
Mean (SD) 37.7 (11.3) 35.6 (11.7) 39.5 (13.3) 34.5 (11.5) 42.7 (12.2) 37.9 (12.1)
Median [Min, Max] 37.0 [20.0, 61.0] 34.0 [19.0, 61.0] 37.5 [20.0, 69.0] 34.0 [18.0, 55.0] 39.0 [27.0, 67.0] 36.0 [18.0, 69.0]
race
Asian 2 (9.5%) 3 (13.6%) 4 (18.2%) 1 (6.7%) 0 (0%) 10 (10.5%)
Black or African-American 4 (19.0%) 2 (9.1%) 4 (18.2%) 4 (26.7%) 0 (0%) 14 (14.7%)
Hispanic/Latinx 1 (4.8%) 0 (0%) 1 (4.5%) 2 (13.3%) 0 (0%) 4 (4.2%)
White 14 (66.7%) 17 (77.3%) 12 (54.5%) 8 (53.3%) 15 (100%) 66 (69.5%)
American Indian or Alaska Native 0 (0%) 0 (0%) 1 (4.5%) 0 (0%) 0 (0%) 1 (1.1%)
gender
Man 11 (52.4%) 13 (59.1%) 8 (36.4%) 7 (46.7%) 4 (26.7%) 43 (45.3%)
Non-binary 1 (4.8%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (1.1%)
Woman 9 (42.9%) 9 (40.9%) 14 (63.6%) 8 (53.3%) 10 (66.7%) 50 (52.6%)
Prefer not to answer 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (6.7%) 1 (1.1%)
matrix_acc
Mean (SD) 0.863 (0.181) 0.835 (0.170) 0.744 (0.274) 0.758 (0.285) 0.600 (0.280) 0.771 (0.248)
Median [Min, Max] 0.875 [0.250, 1.00] 0.875 [0.250, 1.00] 0.875 [0, 1.00] 0.875 [0, 1.00] 0.625 [0, 1.00] 0.875 [0, 1.00]
as.factor(matrix_n_correct)
0 0 (0%) 0 (0%) 2 (9.1%) 1 (6.7%) 1 (6.7%) 4 (4.2%)
2 1 (4.8%) 1 (4.5%) 0 (0%) 1 (6.7%) 2 (13.3%) 5 (5.3%)
3 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (6.7%) 1 (1.1%)
4 0 (0%) 0 (0%) 2 (9.1%) 0 (0%) 1 (6.7%) 3 (3.2%)
5 1 (4.8%) 1 (4.5%) 0 (0%) 1 (6.7%) 4 (26.7%) 7 (7.4%)
6 4 (19.0%) 6 (27.3%) 6 (27.3%) 4 (26.7%) 2 (13.3%) 22 (23.2%)
7 6 (28.6%) 8 (36.4%) 9 (40.9%) 4 (26.7%) 3 (20.0%) 30 (31.6%)
8 9 (42.9%) 6 (27.3%) 3 (13.6%) 4 (26.7%) 1 (6.7%) 23 (24.2%)
0.25
0.5
0.75
1
Overall
high
(N=1)
high
(N=1)
high
(N=2)
high
(N=2)
high
(N=6)
age
Mean (SD) 45.0 (NA) 28.0 (NA) 39.0 (1.41) 21.0 (2.83) 32.2 (10.3)
Median [Min, Max] 45.0 [45.0, 45.0] 28.0 [28.0, 28.0] 39.0 [38.0, 40.0] 21.0 [19.0, 23.0] 33.0 [19.0, 45.0]
race
American Indian or Alaska Native 1 (100%) 0 (0%) 0 (0%) 0 (0%) 1 (16.7%)
White 0 (0%) 1 (100%) 1 (50.0%) 2 (100%) 4 (66.7%)
Black or African-American 0 (0%) 0 (0%) 1 (50.0%) 0 (0%) 1 (16.7%)
gender
Woman 1 (100%) 1 (100%) 1 (50.0%) 1 (50.0%) 4 (66.7%)
Man 0 (0%) 0 (0%) 1 (50.0%) 1 (50.0%) 2 (33.3%)
matrix_acc
Mean (SD) 1.00 (NA) 0.625 (NA) 0.750 (0.177) 0.938 (0.0884) 0.833 (0.171)
Median [Min, Max] 1.00 [1.00, 1.00] 0.625 [0.625, 0.625] 0.750 [0.625, 0.875] 0.938 [0.875, 1.00] 0.875 [0.625, 1.00]
Agent Learning Plots
NonDeviant Analysis
Analysis of Deviance Table (Type II Wald chisquare tests)

Response: corrresp
                                  Chisq Df Pr(>Chisq)    
opinion_round                   33.0185  1  9.129e-09 ***
Deviant_threshold               10.4791  4    0.03309 *  
opinion_round:Deviant_threshold  4.2958  4    0.36746    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
 1       opinion_round.trend     SE  df asymp.LCL asymp.UCL z.ratio p.value
 overall               0.139 0.0237 Inf    0.0924     0.185   5.858  <.0001

Results are averaged over the levels of: Deviant_threshold 
Confidence level used: 0.95 
$emmeans
 Deviant_threshold emmean    SE  df asymp.LCL asymp.UCL z.ratio p.value
 0                  1.537 0.168 Inf     1.207      1.87   9.140  <.0001
 0.25               1.199 0.163 Inf     0.880      1.52   7.371  <.0001
 0.5                1.073 0.160 Inf     0.758      1.39   6.687  <.0001
 0.75               0.898 0.193 Inf     0.521      1.28   4.661  <.0001
 1                  1.560 0.200 Inf     1.169      1.95   7.816  <.0001

Results are given on the logit (not the response) scale. 
Confidence level used: 0.95 

$contrasts
 contrast                                      estimate    SE  df asymp.LCL
 Deviant_threshold0 - Deviant_threshold0.25      0.3382 0.232 Inf   -0.2954
 Deviant_threshold0 - Deviant_threshold0.5       0.4644 0.231 Inf   -0.1644
 Deviant_threshold0 - Deviant_threshold0.75      0.6388 0.254 Inf   -0.0545
 Deviant_threshold0 - Deviant_threshold1        -0.0233 0.259 Inf   -0.7300
 Deviant_threshold0.25 - Deviant_threshold0.5    0.1262 0.227 Inf   -0.4921
 Deviant_threshold0.25 - Deviant_threshold0.75   0.3006 0.251 Inf   -0.3834
 Deviant_threshold0.25 - Deviant_threshold1     -0.3614 0.256 Inf   -1.0589
 Deviant_threshold0.5 - Deviant_threshold0.75    0.1744 0.249 Inf   -0.5057
 Deviant_threshold0.5 - Deviant_threshold1      -0.4877 0.254 Inf   -1.1814
 Deviant_threshold0.75 - Deviant_threshold1     -0.6621 0.276 Inf   -1.4149
 asymp.UCL z.ratio p.value
    0.9717   1.456  0.5913
    1.0932   2.015  0.2589
    1.3320   2.513  0.0876
    0.6835  -0.090  1.0000
    0.7445   0.557  0.9811
    0.9847   1.199  0.7521
    0.3360  -1.414  0.6188
    0.8545   0.699  0.9567
    0.2060  -1.918  0.3078
    0.0908  -2.399  0.1155

Results are given on the log odds ratio (not the response) scale. 
Confidence level used: 0.95 
Conf-level adjustment: tukey method for comparing a family of 5 estimates 
P value adjustment: tukey method for comparing a family of 5 estimates 
Similarity Plot
Similarity Analysis
Type III Analysis of Variance Table with Satterthwaite's method
                              Sum Sq Mean Sq NumDF DenDF F value    Pr(>F)    
targetpair                     528.5   528.5     1    95  2.2408    0.1377    
Deviant_threshold            17044.1 17044.1     1    95 72.2623 2.624e-13 ***
targetpair:Deviant_threshold 14921.1 14921.1     1    95 63.2612 3.768e-12 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
$emtrends
 targetpair Deviant_threshold.trend   SE df lower.CL upper.CL t.ratio p.value
 DN                          -61.32 5.64 95    -72.5   -50.13 -10.878  <.0001
 NN                           -5.56 4.87 95    -15.2     4.12  -1.140  0.2571

Degrees-of-freedom method: satterthwaite 
Confidence level used: 0.95 

$contrasts
 contrast estimate   SE df lower.CL upper.CL t.ratio p.value
 DN - NN     -55.8 7.01 95    -69.7    -41.8  -7.954  <.0001

Degrees-of-freedom method: satterthwaite 
Confidence level used: 0.95 
# A tibble: 2 × 13
  model    term              estimate std.error statistic p.value conf.low
  <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>    <dbl>
1 below_.5 Deviant_threshold   -21.5       9.64    -2.23   0.0292    -40.8
2 above_.5 Deviant_threshold     4.22     11.2      0.376  0.709     -18.3
  conf.high r.squared adj.r.squared    df df.residual  nobs
      <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1     -2.25   0.0733         0.0585     1          63    65
2     26.8    0.00282       -0.0171     1          50    52
ISM Analysis
New Agent Prediction Plot
Prediction Analysis
# A tibble: 2 × 13
  model    term              estimate std.error statistic p.value conf.low
  <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>    <dbl>
1 below_.5 Deviant_threshold   -11.2       17.6    -0.637   0.526    -46.4
2 above_.5 Deviant_threshold    -3.53      14.7    -0.240   0.811    -33.1
  conf.high r.squared adj.r.squared    df df.residual  nobs
      <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1      24.0   0.00640      -0.00937     1          63    65
2      26.1   0.00115      -0.0188      1          50    52
Analysis of Variance Table

Response: confidence
          Df Sum Sq Mean Sq F value Pr(>F)
deviance   4   1469  367.14  0.5063 0.7312
Residuals 90  65258  725.09               
$emmeans
 deviance emmean   SE df lower.CL upper.CL t.ratio p.value
 0          52.4 5.88 90     40.7     64.1   8.914  <.0001
 0.25       55.3 5.74 90     43.9     66.7   9.628  <.0001
 0.5        46.9 5.74 90     35.5     58.3   8.163  <.0001
 0.75       46.8 6.95 90     33.0     60.6   6.731  <.0001
 1          45.0 6.95 90     31.2     58.8   6.472  <.0001

Confidence level used: 0.95 

$contrasts
 contrast                    estimate   SE df lower.CL upper.CL t.ratio p.value
 deviance0 - deviance0.25     -2.8918 8.22 90    -25.8     20.0  -0.352  0.9967
 deviance0 - deviance0.5       5.5173 8.22 90    -17.4     28.4   0.672  0.9620
 deviance0 - deviance0.75      5.5810 9.10 90    -19.8     30.9   0.613  0.9727
 deviance0 - deviance1         7.3810 9.10 90    -18.0     32.7   0.811  0.9267
 deviance0.25 - deviance0.5    8.4091 8.12 90    -14.2     31.0   1.036  0.8381
 deviance0.25 - deviance0.75   8.4727 9.02 90    -16.6     33.6   0.940  0.8806
 deviance0.25 - deviance1     10.2727 9.02 90    -14.8     35.4   1.139  0.7852
 deviance0.5 - deviance0.75    0.0636 9.02 90    -25.0     25.2   0.007  1.0000
 deviance0.5 - deviance1       1.8636 9.02 90    -23.2     27.0   0.207  0.9996
 deviance0.75 - deviance1      1.8000 9.83 90    -25.6     29.2   0.183  0.9997

Confidence level used: 0.95 
Conf-level adjustment: tukey method for comparing a family of 5 estimates 
P value adjustment: tukey method for comparing a family of 5 estimates 
Moderator: Last Opinion
0
(N=21)
0.25
(N=22)
0.5
(N=22)
0.75
(N=15)
1
(N=15)
Overall
(N=95)
pred_maj
Yes 4 (19.0%) 5 (22.7%) 3 (13.6%) 3 (20.0%) 1 (6.7%) 16 (16.8%)
No 17 (81.0%) 17 (77.3%) 19 (86.4%) 12 (80.0%) 14 (93.3%) 79 (83.2%)
# A tibble: 4 × 14
# Groups:   pred_maj [2]
  pred_maj id       term              estimate std.error statistic p.value
  <lgl>    <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>
1 FALSE    below_.5 Deviant_threshold   -10.4       19.0   -0.550    0.584
2 FALSE    above_.5 Deviant_threshold    -5.85      15.1   -0.389    0.700
3 TRUE     below_.5 Deviant_threshold   -26.2       44.7   -0.585    0.571
4 TRUE     above_.5 Deviant_threshold    -2.00      53.2   -0.0376   0.971
  conf.low conf.high r.squared adj.r.squared    df df.residual  nobs
     <dbl>     <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1    -48.5      27.6  0.00590        -0.0136     1          51    53
2    -36.2      24.5  0.00350        -0.0197     1          43    45
3   -126.       73.5  0.0331         -0.0636     1          10    12
4   -139.      135.   0.000283       -0.200      1           5     7
Analysis of Variance Table

Response: confidence
                  Df Sum Sq Mean Sq F value  Pr(>F)  
deviance           4   1469   367.1  0.5073 0.73048  
pred_maj           1   3512  3511.7  4.8522 0.03032 *
deviance:pred_maj  4    228    57.1  0.0789 0.98857  
Residuals         85  61518   723.7                  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
0
(N=21)
0.25
(N=22)
0.5
(N=22)
0.75
(N=15)
1
(N=15)
Overall
(N=95)
pns_med
High 10 (47.6%) 8 (36.4%) 10 (45.5%) 9 (60.0%) 8 (53.3%) 45 (47.4%)
Low 10 (47.6%) 14 (63.6%) 12 (54.5%) 6 (40.0%) 7 (46.7%) 49 (51.6%)
Missing 1 (4.8%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (1.1%)
# A tibble: 4 × 14
# Groups:   pns_med [2]
  pns_med id       term              estimate std.error statistic p.value
  <chr>   <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>
1 High    below_.5 Deviant_threshold     5.20      25.5     0.204   0.840
2 High    above_.5 Deviant_threshold    -2.94      20.5    -0.143   0.887
3 Low     below_.5 Deviant_threshold   -31.5       25.2    -1.25    0.220
4 Low     above_.5 Deviant_threshold    -8.04      20.8    -0.387   0.703
  conf.low conf.high r.squared adj.r.squared    df df.residual  nobs
     <dbl>     <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1    -47.3      57.7  0.00159        -0.0368     1          26    28
2    -45.2      39.3  0.000821       -0.0391     1          25    27
3    -82.8      19.7  0.0439          0.0158     1          34    36
4    -51.1      35.0  0.00646        -0.0367     1          23    25
Analysis of Variance Table

Response: confidence
                 Df Sum Sq Mean Sq F value Pr(>F)
deviance          4   1623  405.75  0.5444 0.7036
pns_med           1    305  304.61  0.4087 0.5244
deviance:pns_med  4   1670  417.48  0.5601 0.6922
Residuals        84  62607  745.33               
Order of deviant across rounds
Opinion Round
0
(N=95)
1
(N=95)
2
(N=95)
3
(N=95)
4
(N=95)
5
(N=95)
6
(N=95)
7
(N=95)
Overall
(N=760)
trialnum
0 18 (18.9%) 18 (18.9%) 16 (16.8%) 16 (16.8%) 10 (10.5%) 14 (14.7%) 12 (12.6%) 12 (12.6%) 116 (15.3%)
1 16 (16.8%) 19 (20.0%) 10 (10.5%) 8 (8.4%) 15 (15.8%) 13 (13.7%) 15 (15.8%) 11 (11.6%) 107 (14.1%)
2 9 (9.5%) 9 (9.5%) 10 (10.5%) 8 (8.4%) 9 (9.5%) 6 (6.3%) 11 (11.6%) 15 (15.8%) 77 (10.1%)
3 12 (12.6%) 10 (10.5%) 18 (18.9%) 15 (15.8%) 11 (11.6%) 17 (17.9%) 12 (12.6%) 14 (14.7%) 109 (14.3%)
4 15 (15.8%) 10 (10.5%) 10 (10.5%) 15 (15.8%) 13 (13.7%) 16 (16.8%) 7 (7.4%) 9 (9.5%) 95 (12.5%)
5 8 (8.4%) 12 (12.6%) 10 (10.5%) 13 (13.7%) 12 (12.6%) 11 (11.6%) 19 (20.0%) 15 (15.8%) 100 (13.2%)
6 10 (10.5%) 8 (8.4%) 8 (8.4%) 12 (12.6%) 12 (12.6%) 11 (11.6%) 10 (10.5%) 7 (7.4%) 78 (10.3%)
7 7 (7.4%) 9 (9.5%) 13 (13.7%) 8 (8.4%) 13 (13.7%) 7 (7.4%) 9 (9.5%) 12 (12.6%) 78 (10.3%)
Demographics (Attention Check)
0
0.25
0.5
0.75
1
Overall
low
(N=15)
low
(N=25)
low
(N=20)
low
(N=15)
low
(N=16)
low
(N=91)
age
Mean (SD) 35.3 (11.3) 39.3 (8.77) 38.6 (12.8) 36.5 (12.3) 34.7 (13.9) 37.2 (11.6)
Median [Min, Max] 33.0 [22.0, 59.0] 37.0 [28.0, 57.0] 37.5 [20.0, 63.0] 38.0 [20.0, 61.0] 27.5 [19.0, 62.0] 35.0 [19.0, 63.0]
race
Asian 2 (13.3%) 2 (8.0%) 4 (20.0%) 3 (20.0%) 3 (18.8%) 14 (15.4%)
Black or African-American 1 (6.7%) 2 (8.0%) 1 (5.0%) 2 (13.3%) 1 (6.3%) 7 (7.7%)
White 12 (80.0%) 17 (68.0%) 14 (70.0%) 9 (60.0%) 11 (68.8%) 63 (69.2%)
American Indian or Alaska Native 0 (0%) 1 (4.0%) 0 (0%) 0 (0%) 1 (6.3%) 2 (2.2%)
Hispanic/Latinx 0 (0%) 2 (8.0%) 1 (5.0%) 1 (6.7%) 0 (0%) 4 (4.4%)
Other 0 (0%) 1 (4.0%) 0 (0%) 0 (0%) 0 (0%) 1 (1.1%)
gender
Man 6 (40.0%) 10 (40.0%) 6 (30.0%) 7 (46.7%) 7 (43.8%) 36 (39.6%)
Non-binary 2 (13.3%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 2 (2.2%)
Woman 7 (46.7%) 15 (60.0%) 14 (70.0%) 8 (53.3%) 8 (50.0%) 52 (57.1%)
Prefer not to answer 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (6.3%) 1 (1.1%)
matrix_acc
Mean (SD) 0.908 (0.0880) 0.960 (0.0595) 0.944 (0.0949) 0.908 (0.0999) 0.898 (0.131) 0.929 (0.0953)
Median [Min, Max] 0.875 [0.750, 1.00] 1.00 [0.875, 1.00] 1.00 [0.750, 1.00] 0.875 [0.750, 1.00] 0.938 [0.625, 1.00] 1.00 [0.625, 1.00]
as.factor(matrix_n_correct)
5 0 (0%) 0 (0%) 0 (0%) 0 (0%) 2 (12.5%) 2 (2.2%)
6 2 (13.3%) 0 (0%) 3 (15.0%) 3 (20.0%) 1 (6.3%) 9 (9.9%)
7 7 (46.7%) 8 (32.0%) 3 (15.0%) 5 (33.3%) 5 (31.3%) 28 (30.8%)
8 6 (40.0%) 17 (68.0%) 14 (70.0%) 7 (46.7%) 8 (50.0%) 52 (57.1%)
0
0.25
0.5
1
Overall
low
(N=1)
low
(N=1)
low
(N=1)
low
(N=4)
low
(N=7)
age
Mean (SD) 38.0 (NA) 35.0 (NA) 34.0 (NA) 40.3 (10.6) 38.3 (7.97)
Median [Min, Max] 38.0 [38.0, 38.0] 35.0 [35.0, 35.0] 34.0 [34.0, 34.0] 40.0 [29.0, 52.0] 35.0 [29.0, 52.0]
race
White 1 (100%) 1 (100%) 0 (0%) 2 (50.0%) 4 (57.1%)
Hispanic/Latinx 0 (0%) 0 (0%) 1 (100%) 0 (0%) 1 (14.3%)
Black or African-American 0 (0%) 0 (0%) 0 (0%) 2 (50.0%) 2 (28.6%)
gender
Woman 1 (100%) 0 (0%) 0 (0%) 3 (75.0%) 4 (57.1%)
Man 0 (0%) 1 (100%) 1 (100%) 1 (25.0%) 3 (42.9%)
matrix_acc
Mean (SD) 0.875 (NA) 0.875 (NA) 1.00 (NA) 0.969 (0.0625) 0.946 (0.0668)
Median [Min, Max] 0.875 [0.875, 0.875] 0.875 [0.875, 0.875] 1.00 [1.00, 1.00] 1.00 [0.875, 1.00] 1.00 [0.875, 1.00]
Agent Learning Plots
NonDeviant Analysis
Analysis of Deviance Table (Type II Wald chisquare tests)

Response: corrresp
                                  Chisq Df Pr(>Chisq)    
opinion_round                   41.4648  1    1.2e-10 ***
Deviant_threshold                3.7914  4     0.4350    
opinion_round:Deviant_threshold  4.9432  4     0.2932    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
 1       opinion_round.trend     SE  df asymp.LCL asymp.UCL z.ratio p.value
 overall               0.142 0.0223 Inf    0.0987     0.186   6.394  <.0001

Results are averaged over the levels of: Deviant_threshold 
Confidence level used: 0.95 
$emmeans
 Deviant_threshold emmean    SE  df asymp.LCL asymp.UCL z.ratio p.value
 0                  1.390 0.215 Inf     0.968      1.81   6.462  <.0001
 0.25               1.266 0.166 Inf     0.940      1.59   7.623  <.0001
 0.5                1.350 0.186 Inf     0.985      1.71   7.250  <.0001
 0.75               1.401 0.217 Inf     0.977      1.83   6.468  <.0001
 1                  0.917 0.203 Inf     0.518      1.32   4.509  <.0001

Results are given on the logit (not the response) scale. 
Confidence level used: 0.95 

$contrasts
 contrast                                      estimate    SE  df asymp.LCL
 Deviant_threshold0 - Deviant_threshold0.25      0.1244 0.271 Inf    -0.615
 Deviant_threshold0 - Deviant_threshold0.5       0.0404 0.284 Inf    -0.734
 Deviant_threshold0 - Deviant_threshold0.75     -0.0112 0.305 Inf    -0.842
 Deviant_threshold0 - Deviant_threshold1         0.4733 0.296 Inf    -0.333
 Deviant_threshold0.25 - Deviant_threshold0.5   -0.0841 0.249 Inf    -0.763
 Deviant_threshold0.25 - Deviant_threshold0.75  -0.1357 0.272 Inf    -0.878
 Deviant_threshold0.25 - Deviant_threshold1      0.3489 0.262 Inf    -0.366
 Deviant_threshold0.5 - Deviant_threshold0.75   -0.0516 0.285 Inf    -0.829
 Deviant_threshold0.5 - Deviant_threshold1       0.4329 0.275 Inf    -0.318
 Deviant_threshold0.75 - Deviant_threshold1      0.4845 0.297 Inf    -0.325
 asymp.UCL z.ratio p.value
     0.864   0.459  0.9909
     0.815   0.142  0.9999
     0.820  -0.037  1.0000
     1.280   1.601  0.4970
     0.595  -0.338  0.9972
     0.606  -0.499  0.9875
     1.064   1.331  0.6719
     0.726  -0.181  0.9998
     1.184   1.572  0.5155
     1.294   1.633  0.4763

Results are given on the log odds ratio (not the response) scale. 
Confidence level used: 0.95 
Conf-level adjustment: tukey method for comparing a family of 5 estimates 
P value adjustment: tukey method for comparing a family of 5 estimates 
Similarity Plot
Similarity Analysis
Type III Analysis of Variance Table with Satterthwaite's method
                              Sum Sq Mean Sq NumDF DenDF F value    Pr(>F)    
targetpair                      12.4    12.4     1    91  0.0557     0.814    
Deviant_threshold            13455.0 13455.0     1    91 60.2861 1.174e-11 ***
targetpair:Deviant_threshold  6012.1  6012.1     1    91 26.9379 1.269e-06 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
$emtrends
 targetpair Deviant_threshold.trend   SE df lower.CL upper.CL t.ratio p.value
 DN                           -52.7 6.15 91    -64.9   -40.49  -8.566  <.0001
 NN                           -12.5 5.24 91    -22.9    -2.07  -2.382  0.0193

Degrees-of-freedom method: satterthwaite 
Confidence level used: 0.95 

$contrasts
 contrast estimate   SE df lower.CL upper.CL t.ratio p.value
 DN - NN     -40.2 7.75 91    -55.6    -24.8  -5.190  <.0001

Degrees-of-freedom method: satterthwaite 
Confidence level used: 0.95 
# A tibble: 2 × 13
  model    term              estimate std.error statistic p.value conf.low
  <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>    <dbl>
1 below_.5 Deviant_threshold    -16.0      11.3     -1.42   0.161    -38.6
2 above_.5 Deviant_threshold    -11.3      11.2     -1.01   0.315    -33.8
  conf.high r.squared adj.r.squared    df df.residual  nobs
      <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1      6.58    0.0335      0.0169       1          58    60
2     11.1     0.0206      0.000577     1          49    51
ISM Analysis
New Agent Prediction Plot
Prediction Analysis
# A tibble: 2 × 13
  model    term              estimate std.error statistic p.value conf.low
  <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>    <dbl>
1 below_.5 Deviant_threshold    -12.1      18.7    -0.647   0.520    -49.6
2 above_.5 Deviant_threshold    -17.1      15.4    -1.11    0.273    -48.2
  conf.high r.squared adj.r.squared    df df.residual  nobs
      <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1      25.4   0.00717      -0.00995     1          58    60
2      13.9   0.0245        0.00456     1          49    51
Analysis of Variance Table

Response: confidence
          Df Sum Sq Mean Sq F value Pr(>F)
deviance   4   1953  488.13  0.7195 0.5809
Residuals 86  58344  678.42               
$emmeans
 deviance emmean   SE df lower.CL upper.CL t.ratio p.value
 0          57.3 6.73 86     43.9     70.6   8.515  <.0001
 0.25       48.0 5.21 86     37.7     58.4   9.222  <.0001
 0.5        50.5 5.82 86     38.9     62.1   8.671  <.0001
 0.75       49.9 6.73 86     36.5     63.2   7.415  <.0001
 1          41.7 6.51 86     28.7     54.6   6.402  <.0001

Confidence level used: 0.95 

$contrasts
 contrast                    estimate   SE df lower.CL upper.CL t.ratio p.value
 deviance0 - deviance0.25       9.227 8.51 86    -14.5     32.9   1.085  0.8140
 deviance0 - deviance0.5        6.767 8.90 86    -18.0     31.6   0.761  0.9411
 deviance0 - deviance0.75       7.400 9.51 86    -19.1     33.9   0.778  0.9363
 deviance0 - deviance1         15.579 9.36 86    -10.5     41.7   1.664  0.4613
 deviance0.25 - deviance0.5    -2.460 7.81 86    -24.2     19.3  -0.315  0.9978
 deviance0.25 - deviance0.75   -1.827 8.51 86    -25.5     21.9  -0.215  0.9995
 deviance0.25 - deviance1       6.353 8.34 86    -16.9     29.6   0.762  0.9408
 deviance0.5 - deviance0.75     0.633 8.90 86    -24.2     25.4   0.071  1.0000
 deviance0.5 - deviance1        8.812 8.74 86    -15.5     33.2   1.009  0.8507
 deviance0.75 - deviance1       8.179 9.36 86    -17.9     34.3   0.874  0.9058

Confidence level used: 0.95 
Conf-level adjustment: tukey method for comparing a family of 5 estimates 
P value adjustment: tukey method for comparing a family of 5 estimates 
Moderator: Last Opinion
0
(N=15)
0.25
(N=25)
0.5
(N=20)
0.75
(N=15)
1
(N=16)
Overall
(N=91)
pred_maj
Yes 3 (20.0%) 7 (28.0%) 6 (30.0%) 4 (26.7%) 4 (25.0%) 24 (26.4%)
No 12 (80.0%) 18 (72.0%) 13 (65.0%) 11 (73.3%) 12 (75.0%) 66 (72.5%)
Missing 0 (0%) 0 (0%) 1 (5.0%) 0 (0%) 0 (0%) 1 (1.1%)
# A tibble: 4 × 14
# Groups:   pred_maj [2]
  pred_maj id       term              estimate std.error statistic p.value
  <lgl>    <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>
1 FALSE    below_.5 Deviant_threshold    -6.11      21.5    -0.285   0.777
2 FALSE    above_.5 Deviant_threshold   -21.6       18.4    -1.17    0.250
3 TRUE     below_.5 Deviant_threshold   -10.7       38.6    -0.276   0.786
4 TRUE     above_.5 Deviant_threshold   -13.2       32.0    -0.414   0.686
  conf.low conf.high r.squared adj.r.squared    df df.residual  nobs
     <dbl>     <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1    -49.4      37.2   0.00197       -0.0224     1          41    43
2    -59.0      15.9   0.0387         0.0104     1          34    36
3    -93.4      72.1   0.00543       -0.0656     1          14    16
4    -82.9      56.4   0.0141        -0.0681     1          12    14
Analysis of Variance Table

Response: confidence
                  Df Sum Sq Mean Sq F value  Pr(>F)  
deviance           4   1981  495.34  0.7465 0.56319  
pred_maj           1   2189 2189.21  3.2993 0.07305 .
deviance:pred_maj  4   2956  739.02  1.1138 0.35591  
Residuals         80  53083  663.53                  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
0
(N=15)
0.25
(N=25)
0.5
(N=20)
0.75
(N=15)
1
(N=16)
Overall
(N=91)
pns_med
High 7 (46.7%) 12 (48.0%) 8 (40.0%) 10 (66.7%) 8 (50.0%) 45 (49.5%)
Low 8 (53.3%) 13 (52.0%) 12 (60.0%) 5 (33.3%) 8 (50.0%) 46 (50.5%)
# A tibble: 4 × 14
# Groups:   pns_med [2]
  pns_med id       term              estimate std.error statistic p.value
  <chr>   <chr>    <chr>                <dbl>     <dbl>     <dbl>   <dbl>
1 High    below_.5 Deviant_threshold   -16.1       32.2    -0.500   0.622
2 High    above_.5 Deviant_threshold   -22.3       21.8    -1.02    0.318
3 Low     below_.5 Deviant_threshold    -9.14      22.9    -0.400   0.692
4 Low     above_.5 Deviant_threshold   -15.5       22.5    -0.687   0.499
  conf.low conf.high r.squared adj.r.squared    df df.residual  nobs
     <dbl>     <dbl>     <dbl>         <dbl> <dbl>       <int> <int>
1    -82.4      50.2   0.00989      -0.0297      1          25    27
2    -67.3      22.8   0.0415        0.00156     1          24    26
3    -55.8      37.5   0.00513      -0.0270      1          31    33
4    -62.1      31.1   0.0201       -0.0225      1          23    25
Analysis of Variance Table

Response: confidence
                 Df Sum Sq Mean Sq F value Pr(>F)
deviance          4   1953  488.13  0.6978 0.5957
pns_med           1    160  160.07  0.2288 0.6337
deviance:pns_med  4   1522  380.55  0.5440 0.7039
Residuals        81  56662  699.53               
Order of deviant across rounds
Opinion Round
0
(N=91)
1
(N=91)
2
(N=91)
3
(N=91)
4
(N=91)
5
(N=91)
6
(N=91)
7
(N=91)
Overall
(N=728)
trialnum
0 16 (17.6%) 13 (14.3%) 14 (15.4%) 16 (17.6%) 12 (13.2%) 9 (9.9%) 9 (9.9%) 12 (13.2%) 101 (13.9%)
1 11 (12.1%) 8 (8.8%) 14 (15.4%) 19 (20.9%) 13 (14.3%) 9 (9.9%) 13 (14.3%) 11 (12.1%) 98 (13.5%)
2 11 (12.1%) 13 (14.3%) 18 (19.8%) 8 (8.8%) 6 (6.6%) 10 (11.0%) 11 (12.1%) 14 (15.4%) 91 (12.5%)
3 14 (15.4%) 14 (15.4%) 7 (7.7%) 5 (5.5%) 7 (7.7%) 12 (13.2%) 15 (16.5%) 15 (16.5%) 89 (12.2%)
4 10 (11.0%) 10 (11.0%) 10 (11.0%) 11 (12.1%) 15 (16.5%) 12 (13.2%) 11 (12.1%) 7 (7.7%) 86 (11.8%)
5 12 (13.2%) 8 (8.8%) 9 (9.9%) 12 (13.2%) 5 (5.5%) 14 (15.4%) 13 (14.3%) 10 (11.0%) 83 (11.4%)
6 8 (8.8%) 10 (11.0%) 9 (9.9%) 8 (8.8%) 19 (20.9%) 13 (14.3%) 7 (7.7%) 13 (14.3%) 87 (12.0%)
7 9 (9.9%) 15 (16.5%) 10 (11.0%) 12 (13.2%) 14 (15.4%) 12 (13.2%) 12 (13.2%) 9 (9.9%) 93 (12.8%)
Things to note
  • The PNS moderator is a median split
Unresolved
  • all good